1. Field
The present invention relates to the field of graphical input devices for interactive computer graphics. More specifically, the present invention relates to the rear screen video detection of multiple objects, such as fingers, on a semi-transparent touch screen used to control interactive computer graphics.
2. Art Background
Broadly, graphical input devices for interactive computer graphics sense the actions of a user. Most such graphical input devices are manipulated by the user. These include the devices commonly associated with personal computers, such as light pens, mice, joysticks, track balls, and keyboards. However, there are also a number of devices that directly sense the actions of a user. Touchscreens with sensors in the display screen are one example of such devices. Myron W. Krueger discloses a number of other devices in his book Artificial Reality II, 1991, that use video cameras to detect the position of a user's hand or body in free space. The VIDEODESK concept described in Krueger's book is also described in U.S. Pat. No. 4,843,568, titled Real Time Perception of and Response to the Actions of an Unencumbered Participant/User, issued to Myron W. Krueger, Katrin Hinrichsen and Thomas S. Glonfriddo Jun. 27, 1989. However, this device requires the user to place his hands on a backlit screen mounted on the upper surface of a desk within the view of an overhead video camera and requires the camera's view of the hands to be unimpeded. As a result, the controller is burdensome and bulky.
The drawing prism presented by Richard Greene at the 1985 SIGGRAPH in San Francisco uses a large transparent prism as a drawing surface. As described in "The Drawing Prism: A versatile Graphic Input Device," Richard Greene, ACM Volume 19, Number 3, 1985, pages 103-110, and in U.S. Pat. No. 4,561,017, titled Graphic Input Apparatus, issued Dec. 24, 1985, , a video camera is arranged to view that surface from an angle such that it can only image the points of optical contact between drawing tools and the surface. However, this device is based on the refraction and total internal reflection of light at a dielectric surface and requires rather precise and expensive optical components and alignment.
Another camera-based system is the Sensor Frame of Sensor Frame Corporation. The Sensor Frame controller uses multiple cameras with intersecting fields-of-view to detect multiple light-occluding objects within a frame. This device is described in U.S. Pat. No. 4,746,770 titled Method and Apparatus for Isolating and Manipulating Graphics Objects on Computer Video Monitor, issued to Paul McAvinney on May 24, 1988. This apparatus is capable of simultaneously detecting the positions of multiple fingers. However, it requires multiple cameras to detect the positions of objects. Further, the device has certain problems with "ghost" images when multiple objects are detected.
It is desirable to provide a graphics input device that provides free-form inputs, commonly referred to as gestures, which directly senses the actions of a user without impeding or otherwise restricting a user by requiring him to wear or hold a device. It is further desirable that the device can sense simultaneous multiple inputs. Further, it is desirable to provide such a system that reduces the requirements of image processing so that it can be implemented real-time on a commonly available computer system. Finally, it is desirable to provide a system that can provide an input suitable for controlling a multi-dimensional application, such as a three-dimensional graphics application.